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Field of the Invention 

The invention relates to communications signal processing and communication 
electronics, and particularly to a method for automatic modulation recognition in 
spectrum monitoring applications. 

Background of the Invention 

The problem of automating radio frequency spectrum monitoring is of much practical 
interest. An important aspect of the spectrum monitoring process concerns the classification 
and identification of individual signals and their sources by evaluating properties of the 
signals such as the modulation format. 

Several methods can be used to tackle the modulation classification problem, as it was 
shown in C. Dubuc, D. Boudreau, F. Patenaude, "An Overview of Recent Results in 
AMR", CRC Technical Memorandum, VPCS #10/98, March 1998 and R. Lamontagne, 
"Modulation Recognition: An Overview", DREO Technical Note, No. 91-3, March 1991. 
Some are based on the decision-theoretic approach, which uses probabilistic models to 
minimize the probability of misclassification errors. These classifiers can achieve very 
good results at signal-to-noise ratios (SNR) as low as 0 dB. However, they assume 
knowledge about some of the signal characteristics (e.g. symbol timing). They were also 
developed for very small digital modulation sets (e.g. BPSK vs. QPSK). Therefore, these 
techniques appear less suitable for a practical modulation classification system. 
Other modulation classification algorithms are based on statistical pattern recognition 
theory. Automatic modulation recognition approaches based on pattern recognition 
techniques, such as neural network classification, have recently attracted much attention. 
A typical idea is to use one or several Artificial Neural Networks (ANN) to process 
measurements of discriminating features. This type of classifiers shows good results with 
simulated signals as reported in R. Lamontagne, "An Approach to Automatic Modulation 
Recognition using Time-Domain Features and Artificial Neural Networks (U) n , DREO 
Report, No. 1 169, January 1993; S.C. Kremer, "Automatic Modulation Recogntion 
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Project - Activity Report (Sept. '97- Dec. '98)", Jan. 1998; E.E. Azzouz, A.K. Nandi, 
Automatic Modulation Recognition of Communications Signals, Klewer Academic Press, 
Boston, 1996, 217 p.; A.K. Nandi, E.E. Assouz, "Algorithms for Automatic Modulation 
Recognition of Communication Signals", IEEE Trans, on Comm., Vol. 46, No. 4, April 
1998, pp. 431-436; and E.E. Azzouz, A.K. Nandi, "Automoatic Modulation Recogntion - 
I & IF, J. Franklin Inst., Vol. 334B, No. 2, pp. 241-305, 1997. However, the 
performance of neural networks in a practical system is highly dependent upon the 
training set. Since neural networks can perform "learning vector quantization", they can 
achieve an efficient class definition over a large multi-dimensional feature space. The 
key issue is to identify a set of meaningful features, and to train the network properly. 
, ; One problem is that the network can become so sharp at recognizing the training vectors 

that it performs erroneously when presented with on-line data that differs only slightly 
from that of the training set. Furthermore, the algorithm designer loses much control on 
the classification algorithm, and has more difficulty in applying a priori knowledge of the 
taxonomy of modulation types. 

Another pattern recognition technique used for automatic modulation recognition is the 
decision tree algorithm. Results from Azzouz and Nandi as reported in E.E. Azzouz, 
A.K. Nandi, Automatic Modulation Recogntion of Communications Signals , Klewer 
Academic Press, Boston, 1996, 217 p.; A.K. Nandi, E.E. Assouz, "Algorithms for 
Automatic Modulation Recognition of Communication Signals", IEEE Trans, on Comm., 
Vol. 46, No. 4, April 1998, pp. 431-436; and E.E. Azzouz, A.K. Nandi, "Automoatic 
Modulation Recogntion - 1 & II", J. Franklin Inst., Vol. 334B, No. 2, pp. 241-305, 1997, 
show that a decision tree can achieve a performance comparable to an ANN classifier 
when using the same features. A comparison of their ANN and decision tree classifiers is 
shown in Table 1 . 
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SNR(dB) 


Decision Tree 


ANN 


5 


61.3% 


60.9% 


10 


87.9% 


88.1% 


15 


94.6% 


96.3% 


20 


94.6% 


96.4% 



Table 1 - Overall success rate comparison between the ANN and the decision tree 
classifiers of Azzouz and Nandi for an ensemble of modulation types. 



Furthermore, a decision tree involves a much lower computational complexity than that 
of a neural network. This is mainly due to the highly hierarchical structure of the tree. 
Only a subset of features is calculated in order to classify a signal, instead of all the set 
for a neural network. Knowing that in both cases, the largest amount of computations 
come from the features, there is an obvious computational gain coming from the use of a 
hierarchical structure. However, the known pattern recognition techniques using a 
decision tree algorithm do not achieve good performance at signal-to-noise rations 
(SNRs) below OdB. 

Summary of the Invention 

It is now an object of the invention to provide a more straightforward and 
computationally simpler method of modulation recognition than neural network based 
methods. 

It is another object of the invention to provide a modulation recognition method which 
directly exploits the fundamental characteristics of the potential signals to evaluate a 
decision tree. 
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It is a further object of the present invention to provide a modulation recognition method 
with a range of successful classification which is extended to lower signal-to-noise ratios 
(SNR), preferably as low as 5 dB. 

It is yet another object of the invention to provide a modulation recognition method for 
identification of the following modulation set: 

Continuous Wave (CW), Amplitude Modulation (AM), Double Sideband 
Suppressed Carrier (DSB-SC), Frequency Modulation (FM), Frequency Shift Keying 
(FSK), Binary Phase Shift Keying (BPSK), Quaternary Phase Shift Keying (QPSK) 
modulations and MPSK/QAM/OTHER 

It is still a further object of the invention to provide a modulation recognition method 
wherein the requirements for a priori knowledge of the signals are minimized by the 
inclusion of a carrier frequency construction step. Accordingly, the invention provides a 
method for estimating the noise floor of a signal, comprising the steps of 

examining the signal for amplitude variations for identifying the signal as one of 
an envelope and non-constant envelope signal; 

estimating the carrier frequency and correcting for carrier frequency errors; and 

categorizing the modulation of the signal. 

When the signal is identified as a constant envelope signal, the step of estimating the 
carrier frequency preferably includes the steps of processing the signal by a Test Fourier 
Transform (TFT) of the input signal, determining the square of the absolute value 
obtained, and searching for the maximum frequency sample. 

In a preferred embodiment, the TFT is with zero padding. Furthermore, the step of 
searching preferably includes a coarse search and a fine search. 

When the signal is identified as a non-constant envelope signal, the step of estimating the 
carrier frequency includes the steps of 
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in a first path, processing the signal by a Fast Fourier Transform (FFT) to produce 
a first output; 

in a second path, passing the signal through a square law non-linearity before 
processing by a FFT to produce a second output; 

in a third path, passing the signal through a fourth law non-linearity before 
processing by a FFT to produce a third output; and 

selecting a maximum energy sample among the first, second and third outputs as a 
normalized frequency estimate; 

whereby the first path is selected when the signal is an AM signal, the second path 
is selected when the signal is a DSB-SC or BPSK signal, and the third path is selected 
when the signal is a QPSK signal. 

Brief Description of the Drawings 

A preferred embodiment of the invention will be further described in more detail with 
reference to the attached drawings, wherein: 

FIG. 1 is a functional flow chart of the decision tree of the preferred embodiment of the 
method in accordance with the invention. 

FIG. 2 shows the squared Discrete Fourier Transform (DFT) coefficients of a QPSK 
signal ataSNR of 5dB; 

FIG. 3 shows the squared DFT coefficients of a FM signal at a SNR of 5 dB; 

FIG. 4 shows an instantaneous frequency histogram for a binary FSK signal at a SNR of 

5dB; 

FIG. 5 illustrates the phase processing for instantaneous frequency calculation; 

FIG. 6 shows a phase processed instantaneous frequency histogram for a FSK signal at a 

SNR of 5 dB; 

FIG. 7 shows a DSB-SC signal; 

FIG. 8 shows a DSB-SC signal with absolute values; 

FIGS. 9 and 10 show typical power spectrums for real and pseudo voice signals 
respectively; 
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FIG. 1 1 shows a comparison between the recognition success rates achievable with prior 
art methods and that of the present method; 

FIG. 12 is an overall success rate comparison with a prior art method; 

FIG. 13 is a performance comparison between a DREO classifier and a classifier of the 

present invention. 

FIG. 14 is a graph illustrating the success rate of the modulation recognition method of 
the invention against the frequency offset for SNR-1 5dB; 

FIG. 15 is a graph illustrating the success rate of the modulation recognition method of 
the invention against the frequency offset for SNR=5dB; 

FIG. 1 6 is a flow chart of the frequency construction methof for constant envelope 
signals; and 

FIG. 1 7 is a flow chart of the frequency estimation method for non-constant envelope 
signals. 

Detailed Description of the Preferred Embodiment 

The first fundamental characteristic that the classifier method in accordance with the 
invention examines is the presence of significant amplitude variations in the observed signal 
received over an Additive White Gaussian Noise (AWGN) channel. This first binary test 
allows an initial discrimination between frequency (analog or digital) modulated and 
amplitude or phase (analog or digital) modulated signals. Since this test is also insensitive 
to frequency errors (within the observed bandwidth constraint), it allows a subsequent carrier 
frequency estimation that takes advantage of the absence or presence of amplitude variations. 
This point is very important, since most of the published work about automatic modulation 
recognition assumes a perfect knowledge of the carrier frequency, and does not disclose any 
methods to acquire this knowledge. 

More detailed information is obtained by applying additional binary tests as described further 
below for estimation of the carrier frequency error. The results of simulations presented in 
Example I below show the good characteristics of the estimation method, for the modulation 
formats in the set {CW, AM, DSB-SC, FM, FSK, BPSK, QPSK, A/PSK-QAM}. 
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A flowchart of the preferred decision tree in accordance with the present invention is shown 
in Fig. 1. It is briefly described in the following paragraph. Its specific components are 
discussed in detail further below. 

The first step of the preferred modulation categorization method determines if there are 
significant amplitude variations in the observed signal. The class of signals with little 
amplitude fluctuations (constant envelope) is easily decomposed, after correction for carrier 
frequency errors, into the classes of unmodulated signals (CW) and FM modulated signals. 
This last class is further split in another step into analog FM and digital FM (FSK). The class 
of amplitude modulated signals (non-constant envelope), after being corrected for carrier 
frequency errors, is readily divided in a further step between the one-dimensional (AM, 
DSB-SC, BPSK) and the two-dimensional (QAM, PSK) signals. The fundamental phase 
characteristics of the one-dimensional signals allow the recognition of the AM signals, from 
the two other formats. The DSB-SC signal vectors having more amplitude variations than 
BPSK signals, they can be identified with a simple test. The remaining signals are classified 
as BPSK signals, although the QPSK signals with constellation points that are members of 
the set 3^/4, 5^4, 7^4] can also be recognized at this stage. 

The next few sections present the different tests used in the decision tree. Each test consists 
in a comparison of a feature extracted from the received signal segment with a fixed 
threshold. The result of this test determines the next branch to be used. 

Constant vs non constant envelope signals 

The first step in the preferred modulation recognition method of the invention is to identify 
the constant envelope signals (CW, FM, FSK). PSK signals are not considered as constant 
envelope signals, since practical PSK signals are band-limited, therefore having a non 
constant envelope. 
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The feature used to identify the envelope variations has been introduced in A.K. Nandi, E.E. 
Azzouz, "Automatic Analog Modulation Recognition", Signal Processing, V. 46, 1 995, pp. 
21 1-222 and is the maximum of the Squared Fourier Transform of the normalized signal 
amplitude. It is defined as 

|DFT(a )|* 

where / is the frequency, DFT( ) is the Discrete Fourier Transform, Isf s is the number of 
samples in the sequence and is the amplitude vector centered on zero and normalized by 
its mean. Mathematically, is expressed as 



where x is the received signal vector. 



This feature is a measure of the information in the envelope and allows the separation of 
constant envelope formats from non constant envelope signals, even PSK signals. In fact, 
PSK signals have a periodical fluctuation in the envelope, corresponding to the symbol 
transitions. These fluctuations will cause high energy coefficients, at the symbol rate, in the 
DFT of the centered normalized envelope. 

The squared DFT coefficients for a QPSK signal at a SNR of 5 dB are shown in Figure 2. 
Note that the highest values correspond to the symbol rate (4 kbauds, in this case). This 
information can therefore be used as an estimate for the symbol rate of the PSK and QAM 
signals. The squared DFT coefficients for a FM signal, also at a SNR of 5 dB a are shown in 
Figure 3. 

For the FM signal, all the coefficients are approximately the same and have a small value. 
Even if the SNR is small, the maximum value of these normalized coefficients can be used 



8 



CA 02260336 1999-02-15 

as an efficient feature to recognize constant envelope signals. It gives better results than 
simply using the variance of the normalized envelope, because is it less sensitive to noise. 
Amplitude modulated signals with low modulation indexes are also characterized with this 
method. 

Frequency Estimation and Correction 

A key point in the proper operation of the decision tree of Fig. 1 is the reliable estimation of 
frequency errors. As indicated in the above, the first test on the presence of amplitude 
variations is insensitive to carrier frequency errors, which allows the subsequent selection 
of frequency estimation steps that take advantage of the information obtained through this 
first test. 

There are then two different carrier frequency estimation steps. If the observed signal is 
classified as constant-envelope, then it is assumed that this signal is unmodulated (CW). The 
estimation method of Fig. 16 is then applied. It involves the identification of the maximum 
energy frequency sample in the frequency domain representation of the signal, which, in the 
case of a baseband CW signal, corresponds to the estimate of the frequency offset as reported 
in A.K. Nandi, E.E. Azzouz, "Algorithms for Automatic Modulation Recognition of 
Communication Signals", IEEE Trans, on Comm. Vol. 46, No. 4, April 1998, pp. 431-436. 
If the hypothesis that the signal is unmodulated is true, the proper frequency offset is 
obtained. If the hypothesis is false, the frequency estimate is much less precise, but the test 
on the unwrapped phase produces a decision in favor of the FM-FSK signals even if a 
residual frequency error still exists. The estimation method includes the steps of processing 
the input signal by a fast Fourier transform (FFT) of the input signal, obtaining the square 
of the absolute value obtained and searching for the maximum frequency sample. The FFT 
is preferably with zero padding. The step of searching preferably includes a coarse search 
and a fine search. 

If the observed signal bears significant envelope fluctuations, the frequency estimation 
method is that of Fig. 1 7. In this case, the implicit assumption is that the signals are not two- 
dimensional, except for QPSK. The observed signal flows in three paths, where its samples 
are either processed directly by an FFT, or passed through a square law or a fourth law 
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nonlinearity, before the FFT. The frequency corresponding to the maximum energy sample 
among the outputs of the three paths is taken as the normalized frequency estimate. If the 
signal is AM, the W= 1 path is selected, if it is DSB-SC or BPSK, the M= 2 path is selected, 
and if it is QPSK, the M= 4 path is retained. If any other form of signal is observed, the 
frequency estimate will be erroneous, which results in a signal with a large phase variance, 
and in a correct classification as a two-dimensional signal. 

Frequency Modulated Signals vs CW 

Once frequency correction has been performed, C W signals are distingui shed from frequency 
modulated signals by examining the time history of the instantaneous phase. Assuming a 
perfect correction of the carrier frequency, a CW signal has a constant phase. The variance 
of the unwrapped phase is a good feature representing the variability of the phase. CW 
signals have a near-zero variance of the unwrapped phase, while frequency modulated 
signals have a high variance. However, the instantaneous phase of a sampled signal is very 
sensitive to noise if the amplitude is small. The variance due to noise on the instantaneous 
phase of a sample will vary with its amplitude. Weak samples will have a higher variance 
on their phase than strong samples, since the phase variations increase as the SNR decreases. 
To avoid this problem, a threshold is set on the amplitude of the signal. The problem can be 
minimized by discarding any phase data for which the corresponding amplitude is below the 
threshold. The higher the threshold is set, the lower the residual phase variance is for a C W 
signal, without affecting the phase variance of frequency modulated signals. A threshold 
equal to the mean of the amplitude of the signal is chosen. A higher threshold increases the 
possibility of discarding an entire noise-free constant envelope signal. The distinction 
between C W and frequency modulated signals is therefore obtained by comparing the value 
of the variance of the unwrapped phase (direct phase) to a phase threshold, for the samples 
bearing an amplitude above their mean. The appropriate phase threshold can be obtained by 
using computer simulations as described further below. 
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Digital vs Analog Frequency Modulation (FSK vs FM) 

The instantaneous frequency distribution can be used to identify the modulation type for 
frequency modulated signals. Using directly the instantaneous frequency histogram as a set 
of features is known in the art as reported in E.E. Azzouz, A.K. Nandi, Automatic 
Modulation Recognition of Communications Signals, Klewer Academic Press, Boston, 1 996, 
2 1 7p.5. However, it is possible to characterize a distribution by its mean, variance, skewness 
and kurtosis coefficients in order to reduce the number of features. Among these parameters, 
the kurtosis coefficient produces good results as reported in[R. Lamontagne, "An Approach 
to Automatic Modulation Recognition using Time-Domain Features and Artificial Neural 
Networks (U)'\ DREO Report, No. 1169, January 1993. This coefficient is defined as the 
fourth normalized moment, centered about the mean, of the instantaneous frequency, defined 
as 

(E[f;{t)]) 

where/(0 is the instantaneous frequency (about the mean) at time t. The kurtosis coefficient 
is a measure of the flatness of the distributions. Since this is usually different for analog and 
digitally frequency modulated signals, it can be used as a feature for distinguishing between 
FSK and FM signals. 

The instantaneous frequency is obtained by computing the phase derivative of the observed 
signal. However, the calculation of instantaneous frequency is very sensitive to noise, since 
the instantaneous frequency is the derivative of the instantaneous phase. The derivative 
operator has a linear gain increasing with the frequency. Therefore, it amplifies higher 
frequency noise, especially the one not in the band of interest, and attenuates the useful 
signal. The instantaneous frequency histogram for a binary FSK signal at a SNR of 5 dB is 
shown in Figure 4. In this case, the instantaneous frequency distribution of a FM signalis 
difficult to recognize. This is reflected in the kurtosis coefficient calculation, which has a 
large value. 
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Since it is known at this point that the signal is frequency modulated, a simple way to 
decrease the effect of higher frequency noise enhancement is to filter the phase signal before 
passing it through the derivative. The phase signal has about the same bandwidth as the 
modulating signal. A simple algorithm roughly estimates the effective bandwidth of the 
phase signal and picks a low-pass filter in a library, with a cutoff frequency slightly higher 
than the signal bandwidth. A very small library of filters can be used. Figure 5 shows the 
phase processing needed before taking the derivative of the phase signal. 

Using this pre-filtering technique, the resulting noise in the instantaneous frequency 
sequence is much less important. As an example, using the same signal as the one used to 
obtain Figure 4, the proposed phase processing is applied with a filter having a cutoff 
frequency 1.5 times larger than the useful phase signal bandwidth. The resulting 
instantaneous frequency histogram is shown in Figure 6. Even for human eyes,it is clear 
that the distribution of this instantaneous frequency signal is easier to recognize from that 
of an FM signal. Two peaks can easily be identified, giving even an idea of the frequency 
deviation. 

In this case the kurtosis coefficient of the instantaneous frequency is very low (approximately 
1.36). It is 2.83 without the phase filtering. FM signals usually have a kurtosis coefficient 
above 2.5. Thus, there is a significant performance gain achievable by filtering the phase 
signal before the derivation. 

Because the decision based on the coefficient of kurtosis is largely insensitive to frequency 
offsets, the discrimination between FM and FSK signals is performed directly on the 
observed signal. This allows a simplification in the frequency estimation. 

One-Dimensional Signals (AM, DSB-SC, BPSK) vs Two-Dimensional Signals 

For the non-constant envelope signals, the second step of the preferred method in accordance 
with the invention is to identify signals which have no information in their phase. In the 
absence of frequency or phase errors, these signals correspond to real baseband signals, such 
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as AM (transmitted carrier), DSB-SC and BPSK signals. These signals can be recognized 
from their absolute centered unwrapped phase sequence, as proposed in R. Lamontagne, "An 
Approach to Automatic Modulation Recognition using Time-Domain Features and Artificial 
Neural Networks (U)" , DREO Report, No. 1 1 69, January 1 993 . Although the absolute phase 
of the real baseband signals normally has a small variance, phase unwrapping is necessary 
in practice because the initial carrier phase is random. However, phase unwrapping is very 
sensitive to noise, particularly for DSB-SC and BPSK signals, due to their 180° phase 
transitions. Consequently, phase unwrapping is undesirable since any phase unwrapping 
errors will seriously increase the variance of the absolute centered phase. 

To avoid phase unwrapping of DSB-SC and BPSK signals, a new quantity is used as the 
method in accordance with the invention, the "absolute" phase, which is defined as 

^(/) = z(j/(r)|+yte(0|). ( 4 ) 

where /(/) and Q(t) are the inphase and quadrature samples at time /. Taking the absolute 
values of the real and imaginary parts of a signal yields a phase sequence between 0 and jt/2 
radians, which allows a significant reduction in the size of the observation space required to 
produce a decision. Figure 7 shows the original DSB-SC signal and Figure 8 shows the 
resulting signal when the absolute value of the real and imaginary parts are taken. 

It can be seen that, for a DSB-SC signal, the variance of the phase of the resulting signal is 
small. This is also true for BPSK and AM signals. For two-dimensional signals bearing some 
phase information, the variance of the absolute phase is high, tending toward the variance 
of a uniformly distributed random variable in the interval [0, nil] radians. To allow a better 
separation of one and two-dimensional baseband signals, a threshold on the amplitude of the 
signal is also used (i.e. the samples below this threshold are discarded). This procedure 
reduces the variance of the phase for one-dimensional signals, without affecting significantly 
the variance for two-dimensional signals. The best results occur when the threshold is set 
equal to the mean of the amplitude of the sequence, as can be seen from simulation as 
described further below. 
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AM vs DSB-SC/BPSK Signals 

BPSK signals can be viewed as DSB-SC signals modulated by a binary Non-Return-to-Zero 
(NRZ) sequence. These signals, unlike AM signals, have jumps of radians in their 
instantaneous phase sequence. The simplest way to distinguish them from AM signals, is to 
directly look at the variance of the unwrapped phase (direct phase). AM signals have a lower 
phase variance than DSB-SC and BPSK, due to the frequent phase jumps of radians. As 
explained before, phase unwrapping is difficult for DSB-SC and BPSK signals. However, 
since the expected variance in the phase is high for these signals, phase unwrapping errors 
are relatively unimportant. Furthermore, AM signals are not likely to produce phase 
unwrapping errors. In this preferred method, a threshold (equal to the mean amplitude) is set 
on the amplitude of the samples, in order to reduce the phase variance for AM signals. ASK 
modulation is a digital form of amplitude modulation (AM). ASK signals are therefore 
classified as AM signals. Further processing would be required to recognize them from 
analog AM signals. 

DSB-SC vs BPSK 

Unlike BPSK signals, DSB-SC signals have envelopes whose amplitudes vary substantially 
over time. Consequently, the variance of the envelope can be used as a feature for 
recognizing BPSK signals from DSB-SC signals. However QPSK signals that have been 
previously classified as real baseband signals will be classified as BPSK signal, since they 
also have an almost constant envelope. 

Binary vs Quarternary PSK Signals 

A side effect of using the absolute phase feature for discriminating between one-dimensional 
and two-dimensional signals, is that some QPSK signals can be classified as one-dimensional 
baseband signals. This happens when the transmitted symbols correspond to the constellation 
points in the set {n/4, 3tc/4, 5tt^4 5 77i/4} radians. In this case, taking the absolute value of the 
real and imaginary parts brings all the symbols to n/4 radians. These signals then mimic the 
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behavior of BPSK signals. One method to differentiate between the binary and the 
quaternary cases is to rotate the signal by 7t/4 radians. If the signal is QPSK, its constellation 
points then correspond to the angles { 7E/2, , 3^/2} radians. Once this rotation is done, the test 
on the absolute phase is repeated. For BPSK signals, the results do not change significantly. 
For QPSK signals, the variance on the "absolute" phase is very high, since the symbols then 
often jump between 0 and it/2 radians. 

The Class of Two-Dimensional Signals 

The class of two-dimensional signals is not further processed in the present method. 

Example I 

Simulations 

To characterize the methods of Figs 1 to 3, 500 simulated signals of each of the modulation 
types have been generated and processed in the Matlab environment. These 500 signals cover 
a range of parameters (e.g. modulation index, symbol rate, pulse shaping filters), as detailed 
in the next paragraphs. A sampling frequency of 48 kHz is used, covering a bandwidth 
slightly larger than the occupied bandwidth of most narrowband communications signals 

Sequences of 1 00 ms (4800 samples) were used as inputs to the classifier. Complex baseband 
signals were used, and carrier frequency and phase errors were simulated. The frequency 
error is random and uniformly distributed over the range [-4.8 kHz, 4.8 kHz], while the 
initial carrier phase is uniformly distributed over [-/z; ^. For digitally modulated signals, a 
random delay, uniformly distributed over [0, 7*J (where T % is the sampling period), was 
applied to simulate symbol timing uncertainties. 
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The Simulation of Specific Signals 

For analog modulation schemes, two types of source signals were simulated. First, a real 
voice signal, band-limited to [0, 4 kHz], was used. The second source signal was a simulated 
voice signal using a first order autoregressive process of the form 

X*] = 0.95- jtf* + (5) 

where n[k] is a white Gaussian noise process. This pseudo- voice signal was further 
bandlimited to [300, 4000 Hz]. The use of a first order autoregressive process to simulate 
voice was proposed in[S.C. Kremer, "Automatic Modulation Recognition Project - Activity 
Report (Sept. '97 - Dec. '98)", Jan. 1998. Typical power spectrums are shown for real and 
pseudo voice signals, on Figure 9 and Figure 1 0 respectively. The major difference between 
the real and the pseudo voice signals is the presence of pauses in the real signal. In the 
pseudo voice signal, the signal is present 100% of the time. This leads to different results, 
since unmodulated (or slightly modulated) 1 00 ms-long sequences can be observed with the 
real voice signal. 

For AM signals, a constant value was added to the source signal. The modulation index was 
uniformly distributed in the interval [50%, 100%]. The modulation index was calculated 
using the maximum amplitude value over the whole source signal. The total length of the 
source signal was about 1 20 seconds for the real signal, and 40 seconds for the pseudo-voice 
sequence. From these source signals, 100 ms-long sequences were extracted randomly. 
Therefore, the observed modulation index for a sequence was less or equal to the chosen 
modulation index. 

For frequency modulated signals, a cumulative sum was used to approximate the integral of 
the source signal. Generic FM signals were simulated using real or pseudo-voice signals, 
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with a modulation index uniformly distributed in the interval [1,4]. The bandwidth occupied 
by these signals was therefore between 16 kHz and 40 kHz, using the approximation 

BW-Tfa + l)/^ (6) 

where p is the modulation index and^ is the maximum source frequency (4 kHz in this 
case). The analog Advanced Mobile Phone Service (AMPS) FM signals were approximated 
using a modulation index of 3. 

Continuous-phase FSK signals were simulated by using filtered A/-ary symbols to frequency 
modulate a carrier. Pager signal parameters were based on observations of real signals, with 
2FSK modulation a bit rate of 2400 bps, a frequency deviation of 4.8 kHz and almost no 
filtering. 4FSK signals were also simulated, using the same frequency deviation and a 
symbol rate of 1200 baud. The 19.2 kbps 2FSK signals from the Racal Jaguar Radio were 
simulated using a frequency deviation of 6 . 5 kHz and a 5 th order Butterworth pre-modulation 
filter, with a cutoff frequency of 9.6 kHz. 

For PSK and QAM signals, symbols were filtered with either a raised cosine or a square-root 
raised cosine function. The selection was randomly performed with equal probabilities. The 
roiloff factors of 20, 25, 30, 35, 40, 45 and 50% were uniformly and randomly selected. For 
all these signals, the symbol rates were chosen randomly between 4, 5, 8, 10, 16 and 20 
kbaud. 71/4-QPSK signals similar to IS-54 signals were also simulated, with a symbol rate 
of 24 kbaud and a square-root raised cosine pulse shaping filter with a rolloff factor of 35%. 

Additive Noise 

The simulated signals were passed through an additive white Gaussian noise channel before 
being classified. Note that no filtering was done at the receiver, so the signal observed by the 
classifier was corrupted by white noise. The noise power was calculated from the knowledge 
of the average power of the modulated signal and of the SNR over the sampling bandwidth. 
This SNR was defined as 
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SNR =— ^— , 

where £ is the signal power, N Q is the white noise power spectral density and F s is the 
sampling frequency equal to 48 kHz. 



For amplitude modulated signals (AM, DSB-SC and SSB), the average power was calculated 
using the whole source signals (real and pseudo-voice). This implies that, for a given 
sequence, the observed SNR might be different from the global SNR. This is especially true 
for DSB-SC and SSB signals, where some segments of the signals may have no power at all. 

Table 2 shows the mean observes SNR for the 200 simulated signals of each modulation 
types and the standard deviation ( ) when the desired SNR is 15 dB. Note the results for 
DSB-SC signals using real voice. 



Modulation Schemes 


Mean SNR (Mnp (dB) 


o(dB) 


CW 


15.0049 


0.0671 


AM (V) 


15.0107 


0.0834 


AM (PV) 


15.0035 


0.0526 


DSB-SC (V) 


4.6926 


14.5438 


DSB-SC (PV) 


14.9865 


0.3982 


SSB (V) 


6.0157 


13.9455 


FM(V) 


14.9999 


0.0619 


FM(PV) 


14.9932 


0.0633 


FM (AMPS) (V) 


15.0011 


0.0644 


FSK (Pager) 


14.9711 


0.0671 


FSK4 


14.9936 


0.0689 


FSK (Jaguar) 


14.9367 


0.0673 


BPSK 


14.9931 


0.0664 


QPSK 


14.9910 


0.0615 


PSK8 


14.9843 


0.0678 


71/4-QPSK (IS-54) 


14.9863 


0.0641 


QAM16 


14.9868 


0.1143 


V: Voice PV: Pseudo Voice 



Table 2 - Observed SNRs (15dB). 
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Example II 

Binary Decision Thresholds 

In the disclosed decision tree, each decision consists in a comparison of a signal feature with 
a threshold. For the simulations, these thresholds have been set using the results of direct 
observations of the feature distributions in a training set of simulated signals having a SNR 
of 5 dB . More optimal thresholds can be obtained using empirical results for real signals. The 
selected thresholds are summarized in Table 3 for the different features. 



Feature 


Threshold 


Maximum of the normalized squared FFT of the centered 
normalized envelope (y^ 


1.44 


Variance of the unwrapped phase (rad) (CW vs. FM/FSK) 


0.16 


Variance of the unwrapped phase (rad) (AM vs. 
DSB-SC/BPSK) 


4.0 


Variance of the "absolute" phase (rad) 


0.144 


Kurtosis of the instantaneous frequency 


2.5 


Variance of the normalized amplitude 


0.25 



Table 3: Decision thresholds used in the simulations. 
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Example III 

Phase Filter Selection for FM/FSK Separation 

As discussed above, lowpass filters are required to eliminate the increased high frequency 
noise resulting when the derivative of the phase is obtained. 1 0 th order lowpass Butterworth 
filters were used in the simulations. For each modulation type, a bandwidth larger than the 
maximum bandwidth of the phase signal was selected. In Table 4, the selected cutoff 
frequencies for the lowpass filters are presented. Note that these cutoff frequencies are not 
very tight with respect to the maximum frequency content of the phase signal. 



Modulation schemes 


Cutoff frequency 


All analog FM signals 


9.6 kHz 


Pager signals (2FSK) 


7.2 kHz 


Jaguar radio signals (2FSK) 


12kHz 


4FSK 


12 kHz 



Table 4: Phase filter cutoff frequencies. 



Example IV 
Classification Results 

An estimate of the performance of the modulation classification method of the invention was 
obtained by using the previously described simulated signals. For each modulation type, the 
500 generated sequences were classified using the preferred method of Fig. 1 . There are 8 
possible outputs from the classification system: CW, AM, DSB-SC, FM, FSK, BPSK, 
QPSK, M>SK/QAM/OTHER (M> 2). Tables 5 to 8 show the classification results for each 
of the simulated modulation types at SNRs of 5 and 10 dB. Tables 5 and 7 indicate the 
classification accuracy when the frequency errors are zero, while Tables 6 and 8 present the 
performance when the frequency error is a random variable, uniformly distributed between 
-4.8 kHz and 4.8 kHz (i.e. ± 10% of the sampling frequency). Comparing these two sets of 
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tables indicates that the frequency estimation and correction steps of the present method 
perform very well, and degrade the overall performance only slightly. 





CW 


AM 


DSB-SC 


FM 


FSK BPSK 


QPSK 


MroK 
QAM 
OTHER 


cw 


98% 


2% 














AM(V) 


55.8% 


43.8% 








0.2% 




0.2% 


AM(PV) 


12% 


88% 














DSB-SC (V) 






43.6% 










56.4% 


DSB-SC (PV) 






100% 












FM(V) 


3.2% 


0.2% 




93.4% 


1% 






2.2% 


FM (PV) 








98.6% 








1.4% 


FM (AMPS) 

(V) 




02% 




96% 


2.8% 






1% 


FSK (pager) 










97.2% 






2.8% 


FSK (Jaguar) 








1.6% 


94.2% 






4.2% 


4FSK 










98.6% 






1.4% 


BPSK 












100% 






QPSK 








0.4% 


0.4% 




7.2% 


94.5% 


/4-QPSK os-M) 








3.8% 


4.2% 






92% 


8PSK 










1.2% 






98.8% 


16QAM 
















100% 


SSB(V) 
















100% 



Table 5: Classification results for no error in the carrier frequency at SNR = 5 dB. 
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jfiSl 




LW 


AM 


DSB-SC 


rM 




npOV 


QPSK 


A/PSK. 
QAM 
OTHER 


cw 


98% 


2% 














AM(V) 


55.070 


43. o% 




n *>o/ 
U.2% 




u.2% 






AM fP\T> 


12% 


QQO/ 

0070 




















j11 oo/ 

42.2% 










57.8% 








IUU% 














2.0% 


U.4% 




74 /0 


1 /o 






2% 


nvi v ^ 








no jco/ 








1 .4% 


rM ^/UVLroJ 
(V) 




A 00/ 

0.2% 




CSC 00/ 

Vo.2% 


2.0% 






1% 


FSK (pager) 










97.2% 






2.8% 


FSK (Jaguar) 








6.6% 


89.2% 






4.2% 


4FSK 










98.6% 






1.4% 


BPSK 












100% 






QPSK 








0.2% 


0.6% 




7.4% 


91.8% 


/4.QPSKas-54) 








4.6% 


3.4% 




1% 


92% 


8PSK 








0.2% 


1% 






98.8% 


16QAM 
















100% 


SSB (V) 




2% 


0.6% 








1.2% 


96.2% 



Table $ : Classification results for random errors in the carrier frequency at SNR = 5 dB. 
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cw 


AM 


DSB-SC 


FM 


FSK 


DOC V 


ADC7 


MPSK 
OTHER 


cw 


100% 
















AM(V) 


58.2% 


41.8% 














AM(PV) 


1 1 .6% 


88.4% 














DSB-SC (V) 






61% 










J7/0 


DSB-SC (PV) 






100% 












FM (V) 


12% 






84.6% 


3.4% 








FM (PV) 








100% 










FM (AMPS) 

(V) 


4.8% 






90.8% 


4.4% 








FSK teaser) 










100% 








FSK fJacuar) 








1.2% 


98.8% 








4FSK 










100% 








BPSK 












100% 






QPSK 














34.2% 


65.8% 


/4-QPSK.0S-54) 








3.6% 


5% 






91.4% 


8PSK 
















100% 


16QAM 
















100% 


SSB (V) 
















100% 



Tabl 



e 7: Classification results for no error in the carrier frequency at SNR - 10 dB. 
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wmm 






CW 


AM 


DSB-SC 


FM 


FSK 


BPSK 


QPSK 


QAM 
OTHER 


cw 


100% 
















AM(V) 


58.2% 


41.8% 














AM (PV) 


11.6% 


88.4% 














DSB-SC (V) 






59.8% 










40.2% 


DSB-SC (PV) 






100% 












FM(V) 


12.6% 






84.6% 


2.8% 








FM (PV) 








100% 










FM (AMPS) 

(V) 


6.2% 






89.6% 


4.2% 








FSK (pager) 










100% 








FSK (Jaguar) 








4% 


96% 








4FSK 










100% 








BPSK 












100% 






QPSK 














33.6% 


66.4% 


/4-QPSKos-54) 








2.4% 


6.2% 




27% 


64.4% 


8PSK 
















100% 


16QAM 
















100% 


SSB (V) 




4.8% 


2.6% 






0.2% 


0.6% 


91.8% 



Tabla 8 : Classification results for random errors in the carrier frequency at SNR - 1 0 dB. 
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A very important result obtained from these simulations is the fact that the classification 
method in accordance with the invention performs well for SNRs as low as 5 dB. Prior- 
art simulations have shown that an abrupt performance degradation occurs between 0 dB 
and 5 dB. This good performance is related to the fact that the binary decision thresholds 
used in the present classifier were determined at a SNR of 5 dB. This performance is as 
good or better than most algorithms based on pattern recognition techniques (neural 
networks), which usually have threshold SNRs on the order of 10 to 15 dB as discussed 
in R. Lamontagne, "An Approach to Automatic Modulation Recognition using Time- 
Domain Features and Artificial Neural Netwoks (U)", DREO Report, No. 1 169, January 
1993. 

In examining the tables, it is important to remember that QPSK signals can be classified 
either as QPSK or AfPSK/Q AM/OTHER (M> 2) signals. Therefore, the probability of 
classifying a QPSK signal in either of these categories is 100%. One of the most 
important results to extract from these tables is the difference between the classification 
of analog modulation signals using real or simulated voice. For these modulations, the 
pauses in a real voice source produce unmodulated sequences. For DSB-SC and SSB 
signals, these pauses produce no signal at all. If the 100 ms preferably used by the present 
classification method is mostly a pause, the signal is classified as noise, i.e. 
AfPSK/QAM/OTHER signal. For AM and FM signals, the absence of a source signal 
produces a C W. Therefore, quiet sequences are classified as CW signals in these cases. 
This result can be easily measured by comparing analog modulated signals using real or 
pseudo-voice sources. Pseudo-voice signals are used to measure the performance of the 
classification method when the signal is modulated. Real voice signals are used to 
measure the expected performance of the system in a real environment. 

Dynamic thresholds could be used in the different tests of the decision tree to improve the 
performance of the classification method. If a good estimate of the SNR is available, it 
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could be used to modify some thresholds. This would be particularly useful in the 
recognition of AM signals from CW signals. If an estimate of the a priori probabilities 
also available, thresholds could be modified to minimize the probability of error. 

The modular nature of the estimation algorithm simplifies its extension to additional 
modulation types. Also, although the disclosed approach implicitly assumes an AWGN 
channel, there are possibilities of extending it to more complex channel models by 
employing additional processing such as blind equalization. 
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Figure 1- 
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Figure 6 
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Figure 10 
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In the foregoing, it was assumed that the noise level was known. However, in practice, the 
variance of the noise must be estimated. Two methods are known in the art, the Level 
Crossing Rate (LCR) method and the Maximum Description Length (MDL) method. 

In the spectrum monitoring context, several channels are observed by a wide-band 
channelisation receiver. The receiver requires knowledge of the noise floor in order to set a 
threshold that will produce a constant false alarm rate assuming the background noise has a 
normal distribution. For simplicity and speed, the receiver must detect the noise from a single 
FFT trace output. The length of the trace will vary since it is inversely proportional to the 
bandwidth of the narrowband channels being monitored. To ensure a fast scanning operation 
of the channelised receiver, it is important to minimise the number of points needed to extract 
the necessary information out of an FFT trace. 

The frequency location and the presence of a signal may not be known by the 
monitoring equipment. 

Very few techniques have been studied to address the general problem of noise floor 
or signal-to-noise ratio estimation [Aus95] [Pau95]. A typical approach is to isolate two 
channels, one with the desired signal and one with noise only, and then estimate the noise 
and signal power independently [Kel86]. Another typical approach is to assume that the 
signal dimension or signal location is known [Sto92]. In spectrum monitoring, these 
assumptions are difficult to meet because the channel allocation is changing, and the 
channels themselves are experiencing an on-off behaviour. Also, in general, it may not be 
possible to take the system off-line in an operational environment for periodical 
measurements. The conventional measurement procedure is lengthy and tedious and the 
noise floor is varying with the environmental conditions. According to the inventor's 
knowledge, only one implemented technique has been presented in the literature to do wide- 
band automatic noise floor estimation in the presence of signals [Rea97]. The technique is 
based on morphological binary image processing operators (similar to rank-order filtering) 
on a binary image of the received power spectrum. Thus it does not process the data directly, 
but the image of the spectrum. No precise performance results are presented in the paper. 
Also, no details are given with respect to the amount of data needed to get an estimate. It 
appears that the technique is computational intensive because it needs to generate a plot of 
the spectrum and to perform 2-D binary filtering. Techniques based on the idea of [Sto92] 
can also be used if an estimate of the signal dimension is provided. However, they would 
require many received vectors and result in a computationally complex procedure. 

This* application provides two techniques that operate on the frequency data to 
provide an estimate of the noise floor level. Performance in terms of occupancy and shape of 
the spectrum will be presented. The method is applicable to fast wide-band scanning. 

Before going into the presentation of the two methods, a few definitions are given. 

Measured Occupancy :The measured occupancy of a sequence composed of M elements 
is defined as the percentage (over M) of the elements above a given 
threshold corresponding to a probability of false alarm due to the 
noise floor power level. 

True Occupancy : The true occupancy is the actual number of signals present in a 
spectrum bandwidth. 
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Noise floor power : The noise floor power level is the average power of the noise if no 
signals are present in a defined assignment band of a given region. 

Channel bandwidth : The nominal channel bandwidth is the bandwidth of an assigned 
channel. 

Signal occupied bandwidth : The signal occupied bandwidth is the bandwidth used by the 

channeliser to estimate the power in a given channel. 



Level Crossing Rate (LCR) 

The level crossing rate concept is used in wave propagation and communication 
channel modelling to obtain statistical information. The LCR is a second-order statistic that is 
time-dependent. It has mainly been use by Lee [Lee69][Lee82] in its work in mobile 
communications. The derivation of the level crossing rate for an analogue signal will follow. 

Let a complex received signal z(r) have uncorrected real x(t) and imaginary y(t) 
components given by 



z(t) = x( t ) + jy(t) = e(t)e«'\ 



0) 



where e(t) and are the envelope (The squared envelope fluctuations are the same) and the 
phase of z{t) given by 



U('); 



6{t) = tan"' 



(2) 



The derivative z(/) of z(J) with respect to t is 

z(t) = l*± = x{t)+jy{t). (3) 
dt 

In [Lee82], it is proven that the four random variables x{t\ y(t), x(f)and y(t) are independent 
real Gaussian random variables with joint probability density function given by 



(2*)V A 



-exp 



(4) 



where cr 2 and ft are the variance of the real and imaginary part of z{t) and z{t) respectively. 
In terms of the derivative of the e(j), it is also shown in [Lee82] that 

,(/)=e(Ocos[0(r)] 
y(t)=e(t)sm[e{t)) 

x{t) = e{t) cos[0(/)]- e{t) 6{t)sin[e{t)] 
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Applying a change of variable, the joint probability density function of 6(t), e(/)and 9{t) 
is given by 



p [ e {t\e{f\eif\e[t)}=-^ 



-exp 



(e>(i).eHlW-(t) + e>( t )) 



(5) 



After the integration of (Xj) and 0[t) from 0 to 2 /rand -oo to » respectively, we obtain 



/>(g(^ <?('))= / *^ exp| 



-2 ^2 



(6) 



The level crossing rate is the total number of crossing per second of a signal at a given 
threshold. It is given by 



o <4Lfco v 2c J 

If the normalised level R defined as R = a/ ^jla 2 is used than we have 



(7) 



PR 



expl 



(-*)• 



(8) 



The variance /?,( ()i ( ) )(o)= 2p 2 of the derivative process z(t) is given in general by 

^^.•(>M.)( r ) 



^i(/)j(<)(0) — 



dr 1 



(9) 



r-0 



The LCR function in (8) is illustrated in 
Figure 3 for several values of the parameter a of the autocorrelation function 
tf r (,) l(/ )(r) = <r 2 exp(-a V). For this function we have p 2 = 2a 2 a 2 . Note that the peak of 
the function is located at -3 dB from the actual variance of the data. It can thus be used to 
estimate the noise floor power when most of the signal is noise. As well, it is observed that 
the LCR value decreases as the correlation parameter 1/a increases. 
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Maximum Description Length (MDL) 

Let's assume that the known covariance matrix R of N FFT trace output vector y„, 
n = 1, 2, .N 9 of length MK is of the form 

^.y^R-S^'l. (10) 

where S is the covariance matrix of the signal components, cr 2 is the variance of the additive 
complex white Gaussian noise in the frequency domain, and I is a unitary diagonal matrix. 
The matrix S can be decomposed into a quadratic form as 

S-VAV", (11) 

where v is a matrix with columns eigenvector v* associated with the ft-th element on the 
diagonal matrix A, the eigenvalue X k . If we assume that the number of signals q in S is 
smaller than MK, than it is readily apparent that the last MK- q eigenvalues, of R will be 
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equal (the eigenvalues of R are given by Xk + a*) to a 2 . The number of signal can thus be 
estimated. For our application, the matrix R is not known and must be estimated. When 
estimated from a finite sample size, the resulting eigenvalues are all different with probability 
one, thus making it difficult to determine the number of signals merely by observing the 
eigenvalues. 

Suppose it is assumed that the signal dimension is k < MK. The parameter vector of 
the model would then be0 ( * } = (V»,^,a 2 f vf,'»,vJ f Ae eigenvalues and 

eigenvectors of the covariance matrix R ( * } . This matrix can thus be expressed as [Max85] 



(12) 



Under the assumption that the vectors y„ are statistically independent, the joint probability 
density function of the N vectors is 



(13) 



Taking the logarithm and omitting terms that do not depend on the parameter vector 0 ( * } , the 
log-likelihood function L(k) is given by 



with 



R = -£y,y w H 



(15) 



The maximum likelihood estimate is the value of ® {k) that maximises (14). These estimates 
are given by [Max85] 



1 K 



(16) 
(16a) 
(16b) 



where h> h> Iak and c x ,...,cmk are the eigenvalues and eigenvectors of (15). 
Substituting (16) in (14) and removing the constant terms, it is found that 



L{k)=N\n 



K 



sK-k 

) 



,* = 0, 1 MK-\. 



(17) 
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The MDL criteria estimate of q is then defined as the value of k that minimises -L(k) plus a 
penalty function related to the number of free parameters in the model 0 ( *>. In mathematical 
terms it is defined as 



<7mdl =argmin|-l(*)+-^(2M/:-*)ln^J. 



(18) 



The index q^ L is the dimension of the signal space. Thus to estimate the noise variance it is 
sufficient to compute 



a' - - 



1 



MK 



(19) 



These prior art concepts presented above are general and are not readily applicable to 
the noise floor estimation for wide-band FFT filter bands. They need to be modified or 
refined to fit with the particularity and the practicality of the real world system. 

The present invention now provides methods that are implemented in a real-time 
system. One problem with the known LCR is that it is not intended for use with a finite 
discrete-frequency sequence. To overcome this difficulty, the present invention provides a 
method which uses a histogram with quantised decibel values. Thus the first step is to 
compute the logarithm values of the envelope squared of the complex FFT" values at the 
output of the filter bank. The exact analysis of the quantized data is difficult because of the 
non-linearity of the operation. To demonstrate that approach still leads to an estimate of the 
actual noise floor level, the analogue signal will be used. Later, examples of the discrete 
approach will be presented. 

The squared amplitude of the filter bank complex values result in a random variable Y 
having an exponential probability density function when only a noise signal is present. 
Taking the logarithm base 10 and multiply by 10 results in a new random variable called 
Z = 10 log jo Y with a probability density function given by 



Pz 



10 



20crMog 10 <? 



exp 



10 



2a 2 



(20) 



with z going from -co to oo. Following the procedure in section 0 where e(t) is now replaced 
by Z, and knowing that it has been shown in [Ric48] that for most wide processes, the 
derivative of Z is independent of Z and Gaussian. The joint probability of the 

process is found to be 



10 



20cr 2 log I0 e 



-exp - 



10 



2cr 2 



exp 



z 2 



2p> 



(21) 



To evaluate the LCR, it suffices to use (21) with the formal definition of the LCR, i.e., 
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n[Z = A]=jzpJz,z)dz 

0 



A/ ( A;' \ 

-exp 



20cr 2 log 10 e 



10 /1Q 

(22) 



2a 



2 

J 



The function (22) has a single maximum that can easily be verified to be located at 
101og l0 (2cr 2 ). The main difficulty in (22), is to evaluate p, the standard deviation of the 
derivative of the logarithm of the envelope squared. The value will depend on the auto- 
correlation function of the envelope process, thus on the window choice. However in the 
present context, this value is not really needed since the value of a* is not even known in the 
first place. The important parameter to estimate is the value of a 2 and it can be estimated 
with the position of the maximum of the function. 

The expression (22) is for a continuous-time signal. To deal with discrete-time signals 
and to ease implementation, (22) is approximated by quantizing the measured dB values to 
the nearest integer value and to build a histogram of the number of occurrences of the 
crossing of the integer dB values. This procedure produces a function having a shape very 
similar to (22) when only noise is present. When a large number of modulated signals are 
present, the global maximum may not be the estimate of the noise power. Figure 1 is showing 
the discrete LCR curves for two scenarios (* Note that the average normalised number of 
occurrence per FFT vector is dependent also on the window choice.) of channel occupancy, 
with the function (22) with noise only. 

The noise power estimate per bin for the noise only scenario is -9 dB while when the 
band is occupied at 75 percent, it would be 15dB if the global maximum is selected. To 
avoid this wrong behaviour, the first local maximum from the right side is selected. To 
ensure that no greater local or global maxima are close to this local maximum, it is then 
verified that there is no greater maximum on the left side closer to Y dB. 

Accordingly the method of the invention comprises the steps of: 

1 ) Computing an FFT amplitude trace output in dB, of an input complex vector, 

2) Quantising the dB value to the nearest integer dB value, 

3) Creating an empty histogram with bins from a minimum value to a maximum 
value, of the quantised B values, 

4) For every positive slope segment of the quantised trace,- adding a 1 to the 
histogram bin that is crossed by the curve, except the end point of the positive 
curve segment, 

5) Finding the first local maximum of the histogram starting from the minimum 
quantised dB value, 

6) Checking if an other local maximum is present K=Z higher than the current 
maximum: if no, than the current maximum abscissa is the noise floor level per 
bins; if yes, going to that maximum and applying 6) again until the answer is no. 
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7) Finding the noise floor per channel by applying the dB correction of 

dB value of noise floor per bin + 10 logio(number of used bin per channel) + 3. 

The method performs very well when the number of bins per channel and the number of 
channels per FFT trace are large and when the occupancy level is relatively low. 

There are two problems with the MDL of the art. The first one is that it requires a 
large number (larger or equal to the number of channels times the number of bins per 
channel) of FFT trace outputs to form a covariance matrix. In a fast channelisation receiver, 
this is unacceptable. Secondly, it requires an eigen-decomposition of a large matrix. This is a 
complex operation that is likely to increase significantly the processor load. Thus is must be 
avoided. Also, it requires a sort function on the eigenvalues which is also a complex 
operation. These problems are now overcome with, a simplified MDL method in accordance 
with the invention. 

The first requirement is that the noise power level be determined from a single FFT 
trace. To avoid the sort operation, the squared FFT coefficients of a trace are transformed in 
dB values and then quantised to the nearest integer values. This operation is needed anyway. 
The interger values are then put in an histogram and subsequently extracted from the 
histogram by group of AT, the number of bins per channel. This is effectively the equivalent of 
a sort function and a very good approximation to the true sort operation. It also reduces the 
side of the problem from MK to M. An other impact of the operation is that the sample power 
will tend to be decorrelated even with a window because they are now from non adjacent 
bins. Next as in the case of the MDL, these M points corresponding to the power of K bins in 
decreasing order are used in the log-likelihood function of (14). The penalty function />(£) is 
subtracted from the log-likelihood function to form the criteria over which the minimum will 
be searched. The penalty function however in this case is a different polynomial function 
compared to the one of the prior MDL. The reason for the change is due to the fact that the 
method of the invention does not need to account for the eigenvectors, thus reducing the 
penalty of interested is only the amplitude of the equivalent of the eigenvalues. Thus when 
% « o, we have MN free parameters, when k = 1 , we have (M-l)N free parameters and so on. 
The penalty function is however a second order polynomial. It was determined by examining 
the log-likelihood function when only noise is present and fitting a polynomial function. This 
polynomial function is then scaled to avoid erroneous detection due to the variance in the 
data. Thus the only assumption here is that the FFT coefficients have a Gaussian distribution 
which should be valid since they result from a linear combination of variables that are 
Gaussian. If the sample data are not normal, the linear combination will then transform then 
in Gaussian variables. The scale factor gives a degree of freedom in the optimisation process. 
Figure 2 illustrates the log-likelihood function and the fitting function. The actual function 
that is minimised is 

g w " = argminM*)+C/>(4- ( 23 ) 

In another aspect, the invention provides a method including the steps of: 

1) Building a histogram from a FFT output trace quantized to the nearest integer dB 
value, 

2) Transforming the histogram into a sorted linear vector, 
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3) Chanelizing the sorted linear vector into M groups, by summing the samples of 
the sorted vector, 

4) Applying the log-likelihood function on the sorted group, of M elements 

5) Adding the penalty polynomial function to the negative of the log-likelihood, 

6) Finding the index of the global minimum, and 

7) Estimating the noise floor as the average of the last M minus the index number 
smallest sorted groups. 

The performance of both methods in accordance with the invention is a very 
important factor but it must also be weighed against the complexity of the approaches. The 
starting point will be the MK values of the complex squared FFT coefficients and the 
stopping point will be after the noise floor estimate per FFT bin. The complexity will be 
measured by counting the number of multiply, add, compare and math functions. 

Table 1 summarises the operations for both methods. From that table we conclude 
that both methods are approximately of the same complexity. Both methods operate on the 
bin basis and can provide the noise floor per bin or channel. Both methods are dominated by 
the compare operations. 



Table 1, Complexity comparison of the discrete LCR and simplified MDL. 





LCR 


MDL 


LCR 256. AT = 8) 


MDL (A/ = 256,AT=8) 


x/ 


MK 


MK + 6M 


2048 


3584 


+ - 


2MK 


MK + 4M 


4096 


3092 


log 10 or log* 


MK 


MK+M 


2048 


2304 


Compare 


*3MK + 60 


2MK+M 


6204 


4352 


Round 


MK 


MK 


2048 


2048 


cr 


0 


*A/+60 


0 


316 


Total 


~ ZMK 


* UMK+ 13 M 


16444 


15696 



Example I 

To evaluate the respective performances of both methods, a simulation program and a 
real-time implementation were written to estimate the mean and variance of the noise floor 
level computed by the LCR and MDL. The mean value of the estimate will indicate if a bias 
is present and its size in the algorithms while the variance will indicate the stability of the 
estimate. 

The transmitted signals used for the simulation are a binary FSK signal similar to a 
Jaguar radio (20 kbps data rate instead or 19.2 kbps for the Jaguar radio), and a measured FM 
signal in the 139 to 144 MHz band. The channel bandwidth is 25 kHz for the binary FSK 
signal and 15 kHz for the FM signal. The windowed FFT filter bank is using a Blackman 
window. To reduce the amount of simulation results to present, the number of channel M has 
been limited to 64 and the number of bin per channel K to 8. These numbers are typical of 
what is expected in practice. Table 2 presents the results for other choices of M and K for an 
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occupancy level of 50 % and the equal carrier power pattern, where it is apparent that for 
K > 4, the results do not change very much as a function of M and K. 



Table 2. Mean estimate value for 50 % occupancy of jaguar signals with 
equal carrier power at 20 dB above 0 dB noise floor. 



K 


A/= 16 


Af=64 


A/=256 


1 


MDL= 51.5; LCR = 68.2 


MDL = 47.5; LCR = 37.5 


MDL = 48.5; LCR = 47.6 


2 


MDL =27; LCR = 29.4 


MDL = 10.4; LCR - 7.93 


MDL = 3.07; LCR= 12.7 


4 


MDL- 1.4; LCR » 4.7 


MDL = 1.0 1; LCR = 1.3 


MDL = 1.05; LCR = 1.93 


8 


MDL = 1.12; LCR = 1.14 


MDL = 1.13; LCR= 1.35 


MDL = 1. 16; LCR = 1.56 


16 


MDL = 1.17; LCR = 1.12 


MDL =1.17; LCR= 1.39 


MDL = 1.2; LCR = 1.47 



The results of the estimators are presented for occupancy level of 0 % to 93.75 % 
with four different carrier power patterns. They are referred to as equal power (dBe), ramp 
power (dBr), ramp and equal power (dBer), and equal and ramp power (dBre). The typical 
shapes are illustrated in Figure 4. These are the pattern if the carrier power were sorted. In 
the simulation, the amplitudes are assigned random frequency locations multiple of the 
channel bandwidth. The results are illustrated in Figure 5 to Figure 9 for the best parameters 
of the two methods for the four power patterns of Figure 4. From the above results, it is 
apparent that both methods have less than a 3 dB bias error at up to about 60% occupancy. 
The bias could potentially be removed since it is possible to have an estimate of the 
occupancy on the trace. The bias appears because the Jaguar signal (binary FSK signal) is a 
difficult signal in the sense that it fills its 25 kHz assigned bandwidth completely (see Fig. 
16). For signals where a guard band is present, the performance will in general be even better 
than shown in Fig. 5 for the most difficult cases of an FM signal, i.e. equal carrier power 
level. From these sets of results, it is concluded that the LCR has slightly better general 
performance but with a larger variance than the MDL. For the FM signal, the MDL seems to 
have an advantage. Next Fig. 10 presents the time variations of the estimates of the two 
methods for 50% occupancy for 64 channels and 8 bins/channel with the binary FSM signal. 
A conclusion from these figures is that time averaging of the noise level estimate will be 
necessary to limit the false detection rate. This should not be a problem since the noise 
process is usually a very slow varying process. 



FM signal at 20 dB SNR with equal power levels. 

Example II 

DAS Measurements 

The results of the previous section were indicative of the bias and the variance in the 
estimators. The objective in the estimation of the noise floor is to derive the threshold level 
for detection. This section presents results on the average probability of false alarm of the 

unused channels \ and the probability of detection of one channel for the two noise floor 
level estimation methods, and for the known noise floor level as a comparison. The desired 
signal is at the maximum power level of the power pattern. The impact of the estimator's 
variance are assessed for the equal-ramp power pattern as a function of the time average of 
the estimate, the occupancy level, and the signal-to-noise ratio. The signal-to-noise ratio is 
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defined as the amplitude square of the transmitted signal over the noise variance in the 
nominal channel bandwidth. For a given occupancy level, the power pattern is fixed and 
relative to the maximum SNR, i.e. 30 dB. This means that as the SNR increases, more 
channels are visible out of the noise floor. The synthesised signal is a BFSK radio in a 
25 kHz spacing with M = 64 channels, N « 6 bins used for detection out of K « 8 bins in the 
detection process, a desired P& = 10"\ and number of averages of 1, 10, and 100 estimates. 
The results are illustrated in Figures 1 1 to 15. 

Figure 1 1 shows that /> fa increases as a function of the SNR and the occupancy for the 
known SNR case. This is not a surprise since the transmitted signal causes adjacent channel 
interference. The occupancy level does not affect the probability of detection. For the noise 
floor estimators now, Figure 12 presents the probabilities when the instantaneous values of 
the estimators are used in the threshold. The LCR has an order of magnitude increase in its 

P H relative to the desired value. The MDL is close to 10" 3 for low occupancies but is below 
for moderate and high occupancy for the range of SNR below 20 dB. This is indicative that 
the algorithm over-estimates the noise floor level. The probability of detection decreases as a 
function of occupancy and the LCR seems to provide a few fraction to a few dB of advantage 
over the MDL in the SNR range from 0 to 12 dB. Figure 13 is for the same scenario as 
Figure 12 except that 10 averages are used instead of 1. The behaviour of P& is similar to the 
case of a single average of the noise floor estimate. The average probability of false alarm of 
the unused channels of the LCR and MDL is now similar with the LCR values being higher 
in the range of SNR up to approximately 22 dB. The results of averaging 100 estimates are 
presented in Figure I t and they are basically the same as Figure 13. This means that 
averaging 10 estimates is sufficient to guarantee a "normal" behaviour compared to the 
known SNR case. Finally, Figure 1 5 presents the results for the FM signal were it is seen that 
in the absence of weak adjacent channel interference, the difference between the known 
channel SNR and the estimated noise floor performances are negligible. 

It should be pointed out that for strong signals, the probability of false alarm of s 
unused channels is always larger than the desired value. These results should be taken into 
account in the detection algorithm or in a database storing the power results for further 
analysis. 

The following parameters are preferred for carrying out the first version of the noise 
floor estimation: Implement the LCR algorithm with 2 dB look over, 

a) A window of 10 estimates is sufficient to reduce the time variation of the 
estimate to an acceptable level of probability of detection degradation, 

b) Measure the LCR performance in terms of variance and bias as a function of 
the measured occupancy (estimated with a threshold), 

c) Implement and test the MDL in a test version (in the DAS or in Matlab) to 
verify the observation that ii performs well with real life signals (Figure 9 and Figure 15) 

d) Investigate the possibility of reducing the complexity of the algorithms. 

These recommendations together with the results of the measurements should then 
indicate if further refinements to reduce the bias as a function of occupancy are really needed 
for real life signals. Also, a mechanism to indicate the failure of the algorithm should be 
developed so that bad estimation should be marked. 
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Crossing quantised values [dB] 
Figure 1. Example of discrete LCR curves for 256 channels filter bank with 4 bins 
per channel with a Blackman window. 
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Figure 2. Normalised log-likelihood function and polynomial fit with noise 
only for Af=64 andK= 8. 
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Figure 3. Normalised LCR curve for the complex Gaussian noise only signal. 
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Figure 4. Pattern of carrier signal power. 
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Figure 5. Noise floor level estimation for dBe power, K= 8 , M= 64, and BFSK signal. 
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Figure 6. Noise floor level estimation for dBr power, K= 8 , M= 64, and BFSK signal. 
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Figure 7. Noise floor level estimation for dBre power, A> 8 , M= 64, and BFSK signal. 
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Figure 8. Noise floor level estimation for dBer power, K- 8 , M= 64, and BFSK signal. 
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Figure 9. Noise floor level estimation for dBe power, K= 8 , Af = 64, and FM signal. 
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Figure 10. Noise floor estimate variations at 50 % occupancy with K « 8 and M= 64. 

(a) BFSK signal at 10 dB SNR with equal power levels, 

(b) BFSK signal at 20 dB SNR with equal-ramp power levels, 
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Figure 11. P h and ? d for the BFSK signal with equal-ramp power, 64 channels, 6 bins of out 

8 for detection, and known SNR. 
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Figure 12. P h and for the BFSK signal with equal-ramp power, 64 channels, 6 bins of out 
8 for detection, and 1 average of the noise floor estimate. 
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Figure 14. P h and Pi for the BFSK signal with equal-ramp power, 64 channels, 6 bins of out 
8 for detection, and 100 averages of the noise floor estimate. 
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Figure IS. P h and ?d for the FM signal with equal-ramp power, 25 % occupancy, 64 
channels, 6 bins of out 8 for detection, and 10 averages of the noise floor estimate. 
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Frequency [kHz] 

Figure 16. Average spectrum of BFSK and FM signals with 25 kHz and 15 kHz nominal 
channel bandwidth respectively 
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CLAIMS 

1 . A method for recognizing the type of modulation of a signal, comprising the steps 
of 

examining the signal for amplitude variations for identifying the signal as one of 
an envelope and non-constant envelope signal; 

estimating the carrier frequency and correcting for carrier frequency errors; and 
categorizing the modulation of the signal. 

2. The method of claim 1 , wherein when the signal is identified as a constant 
envelope signal, the step of estimating the carrier frequency comprises the steps of 

processing the signal by a Fast Fourier Transform (FFT) of the input signal; 
obtaining the square of the absolute value obtained; and 
searching for the maximum frequency sample. 

3. The method of claim 2, wherein the FFT is with zero paddng. 

4. The method of claim 2, wherein the step of searching comprises a coarse search 
and a fine search. 

5. The method of claim 1, wherein when the signal is identified as a non-constant 
envelope signal, the step of estimating the carrier frequency comprises the steps of 

in a first path, processing the signal by a Fast Fourier Transform (FFT) to produce 
a first output; 

in a second path, passing the signal through a square law non-linearity before 
processing by a FFT to produce a second output; 

in a third path, passing the signal through a fourth law non-linearity before 
processing by a FFT to produce a third output; and 

selecting a maximum energy sample among the first, second and third outputs as a 
normalized frequency estimate; 
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whereby the first path is selected when the signal is an AM signal, the second path 
is selected when the signal is a DSB-SC or BPSK signal, and the third path is selected 
when the signal is a QPSK signal. 

6. The method of claim 2, including the further step of dividing the signal into 
Continuous Wave (CW) and Frequency Modulated (FM) signals by obtaining a value of 
the variance of the unwrapped phase (direct phase) for signal samples above a preselected 
threshold; and classifying the signal as a C W signal when the variance is substantially 
zero and as a FM signal when the variance is significantly above zero. 

7. The method of claim 6, wherein the threshold is equal to a mean of an amplitude 
of the signal. 

8. The method of claim 7, wherein a FM signal is identified as one of a digital and 
analog frequency modulated signal by determining the kurtosis coefficient of the 
instantaneous frequency distribution of the signal, the kurtosis coefficient being the 
fourth normalized moment, centered about the mean, of the instantaneous frequency of 
the signal, and 

classifying the signal as an analog frequency modulated signal when the 
coefficient is at least 2.5 and as a digital frequency modulated signal when the coefficient 
is below 2.5. 

9. The method of claim 8, wherein the instantaneous frequency is obtained by 
computing the phase derivative of the signal and the method comprises the additional step 
of, prior to computing the instantaneous frequency, filtering the phase signal. 

1 0. The method of claim 9, wherein the step of filtering comprises 
estimating an effective bandwidth of the phase signal; and 

filtering the phase signal with a low-pass filter having a cut-off frequency above 
the bandwidth. 
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1 1. A method for estimating the noise floor of a signal, comprising the steps of 
determining a FFT trace output from the signal; 

quantifying the value of the output to the nearest integer dB value; 

creating an empty histogram with bins from a minimum value to a maximum 

value; 

for every positive slope segment of the histogram curve adding 1 to the histogram 
bin which is crossed by the curve, except the end point of the curve segment; 

finding a first local maximum of the curve, starting from the lowest dB values; 
checking if another local maximum is present XdB higher than the current maximum; 

setting the current maximum as the noise floor position if no other local 
maximum is present XdB higher than the current maximum; 

repeating the step of checking until no other local maximum is present; 

finding the noise floor per channel by applying a dB correction of dB value of 
noise floor per bin + 1 0 log 10 (number of bins used per channel) + 3. 

12. A method for estimating the noise floor of a signal, comprising the steps of 
building a histogram from a FFT output trace quantified to the nearest integer dB 

value; 

transferring the histogram into a sorter linear vector; 
channeling the sorted linear vector into M groups; 

adding the penalty polynomial function to the negative of the log-likelihood; 
finding an index of the global minimum; 

estimating the noise floor as the average of the last M minus the index number 
smallest sorted groups. 



